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Team based learning in the form of its readiness assurance test (RAT) application 
is a popular instructional intervention in health professions education. This 
investigation was conducted to examine the extent to which individual and team 
readiness assurance tests correlate with conventional assessment outcomes in 
pharmacy calculations. Students encountered IRAT and TRAT assessments on the 
Blackboard learning management platform during their pharmacy calculations 
laboratory course. The RAT assessments addressed topics concurrently with 
examination topics in the pharmacy calculations lecture course. Students were also 
assigned self-paced online module assessments pertaining to each lecture topic. 
Both sets of student lecture assessment scores were compared to their respective 
RAT performances using Spearman’s rank correlation measurements. Low-to- 
moderate correlations were observed between students’ IRAT scores and lecture 
course assessment scores, which includes examinations and online modules. 
Several correlations were statistically significant among topics. Very low-to-low 
correlations were observed between students’ TRAT scores and lecture course 
assessment scores. IRAT scores better correlated to final exam performances than 
TRAT scores. RAT assessments, particularly IRAT, may serve as indicators of 
subsequent assessment performance in pharmacy calculations. Use of RAT may 
also assist students in their preparations for comprehensive assessments in the 
subject. 
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INTRODUCTION 


The rise of active learning in education signals an acknowledgement of the importance 
and effectiveness of student-centered approaches to instruction. Student-centered 
instruction accounts for variations in student learning styles, information processing, 
work methods, and use of learning resources. Student-centered learning particularly 
flourishes in the instance of collaboration among students, which promotes increased 
engagement, increased confidence, and better behaviour in students (Burns et al., 2014). 
Student collaboration has been associated with the development and retention of critical 
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thinking skills, and has been noted to positively impact students’ cognitive development, 
emotional awareness, motivation to learn, and receptiveness to novel ideas (Warsah et 
al., 2021). Collaborative learning also promotes student reflection and encourages 
personal assessment of strengths and weaknesses through acceptance of constructive 
criticism from peers (Yaacob et al., 2021). Use of the team construct in the classroom 
has gradually evolved as academia has gained greater appreciation of its value. 


An especially well-structured form of the team construct is team-based learning (TBL), 
which has gained great favor in higher educational instruction, and has proven beneficial 
to health professions education. Team-based learning has also been touted for its faculty 
facilitation, its small group sizes, and its use of immediate feedback (Burgess et al., 
2017). Although TBL encompasses multiple techniques of content delivery and 
reinforcement, a popular feature of this method is use of the readiness assurance test 
(RAT), which is deemed both a vehicle for rapid feedback (Bauler et al., 2020) and a 
motivator of student preparation (Koh et al., 2019). The RAT is distributed to 
individuals or to student teams. Use of RAT assessments allow students to remediate 
through facilitator and team feedback, and through team assessment. The RAT allows 
for identification of weaker performers prior to major assessment. Stronger performers 
are spared from sacrificing scoring opportunities for the sake of team participation, 
which has traditionally been a disadvantage of group work (Parmelee et al., 2012). 
Technology has added versatility to the RAT, as audience response systems such as 
iClicker™, and web-based platforms such as Blackboard™, Moodle™, ExamSoft™ and 
Nearpod™ can be used to manage and collect assessment data efficiently and rapidly 
(Shaikh et al., 2017). 


Context and Literature Review 


Team-based learning is used in the instruction of various pharmacy-related disciplines, 
including pathophysiology, pharmacology and medicinal chemistry. It has been 
favorably received among students and faculty of the basic sciences, and its use has been 
associated with improvements in students’ content mastery (Kolluru et al., 2012). Use of 
TBL has also yielded equalization of pharmacotherapy learning outcomes with 
traditional lecture methods (Wilson et al., 2019), or improved pharmacotherapy 
outcomes compared to traditional methods (Letassy et al., 2008). Improvements in 
students’ learning outcomes have also been reported in the areas of pharmacy 
informatics (Hincapie et al., 2016), pharmacokinetics (Persky, 2012), and pharmacy 
practice (Orr et al., 2015; Zingone et al., 2010). 


Pharmacy calculations is a subject that covers a range of topics that are inevitably 
referenced in pharmacotherapy. Due to its ubiquitous presence throughout pharmacy 
curricula, pharmacy calculations has received renewed interest, appreciation and 
emphasis in recent years (Hegener et al., 2013). This revival of sorts has coincided with 
technology-driven, alternative methods by which students are encouraged to consume 
and to retain course content (Mastropieto et al., 2019; Davies et al., 2018; Bell et al., 
2017; Nutan & Demps, 2014; Lacroix et al., 2014;). Despite this renaissance, there have 
been few published studies featuring the use of TBL in pharmacy calculations 
(Mastropieto et al., 2019; Wara et al., 2019; Davies et al., 2018; Anderson et al., 2017), 
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which includes even fewer references to the use of RAT during its instruction 
(Mastropieto et al., 2019; Wara et al., 2019). Lack of application of TBL in pharmacy 
calculations is counterintuitive, since the method is reported to positively impact the 
instruction of various math-based disciplines. For instance, TBL promotes decreased 
anxiety and increased confidence in collegiate math students (Naughton et al., 2020), 
and encourages positive active engagement among peers (Paterson et al., 2013). Use of 
TBL has also been shown to strengthen students’ problem-solving skills and listening 
skills, along with their abilities to adapt, to innovate and to think independently in 
STEM (science, technology, engineering and math) subjects (Hashim et al., 2018). 
Given the effectiveness of RAT exhibited in other areas of pharmacy and in math 
disciplines, the applicability of the learning tool to pharmacy calculations is intriguing. 


This investigation was conducted to determine the extent to which readiness assurance 
testing correlates with traditional learning assessments in pharmacy calculations. The 
investigation was performed during a single semester with Ist year professional program 
students. Pharmacy calculations was offered in the form of a lecture course and an 
adjoining laboratory course. The lab course offered reinforcement of topics introduced 
during lectures. Reinforcement was provided through hands-on activities, recitation and 
assessment. Whereas TBL was a challenge to execute during the lecture period due to 
room logistics (ie., inconvenient auditorium seating) and time constraints, the lab class 
was a more conducive setting. Lab student groups assembled at large tables. In addition, 
students were enrolled into one of three lab course sections, which featured substantial 
time intervals for activities. Each lab section met once per week. 


METHOD 


Study procedures were approved via Institutional Review Board oversight. Students (n = 
69) in the pharmacy calculations lecture course were assessed using four written 
examinations and a stand-alone quiz. Exam 1 focused on the topics of prescription 
interpretation, the metric system, and intra- and inter-system unit conversion. Exam 2 
focused on the topics of dosage calculations, reducing and enlarging formulas and 
specific gravity. Exam 3 focused on the topics of expressions of strength, dilution and 
concentration, and intravenous admixtures/ infusions. The topic of chemical equivalency 
was taught and tested upon as a quiz subsequent to the first 3 examinations. A 
comprehensive final examination was given at the culmination of the course. Students 
in the lecture course were also assigned online practice modules as graded homework. 
The modules were available on the Blackboard learning management system, and were 
released to students in conjunction with topics introduced in lecture. Module exercises 
were un-proctored, and were available for student use any time of day until the end of 
the lecture. They could be encountered with limitless repetition until students attained 
scores to their satisfaction. First attempt scores were recorded, and highest scores 
attained during each set of module attempts were incorporated into students’ final 
grades. 


Pharmacy calculations lab students formed in groups of 3 for the TBL activities. All 
students were exposed to all readiness assurance test interventions. Readiness assurance 
test activities were performed using the Blackboard learning management system, which 
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provided an efficient platform for question randomization and dissemination, and for 
record keeping. Students were encouraged to use their personal laptops and tablets 
during assessments. The Respondus Lockdown Browser™ application on Blackboard 
was mandated for assessment access. The lockdown browser secured the assessments 
by disallowing students from opening other websites or personal files. Assessments were 
proctored using a faculty member and/or graduate teaching assistants assigned to each 
lab section. Readiness assurance test procedures began with a 20-minute individual 
readiness assurance test (IRAT). Students then retreated to their groups to take a 20- 
minute team readiness assurance test (TRAT), which included question stems that 
matched those of the IRAT questions. Students were provided a brief review period 
after the TRAT session, during which time they were able to obtain clarification about 
areas of uncertainty. Students who either did not possess electronic devices or 
experienced technical difficulties during assessments were provided alternative paper 
assessments for the readiness tests. Topics encountered during IRAT and TRAT 
sessions matched the sequence of those presented during the pharmacy calculations 
lecture course. Conceptual material addressed during the weekly IRAT and TRAT 
included: 


e IRAT/ TRAT 1-- Metric system and unit conversions 
e IRAT/ TRAT 2—Dose calculations 

e IRAT/ TRAT 3—Reducing and Enlarging Formulas 
e IRAT/ TRAT 4—Specific gravity 

e IRAT/ TRAT 5—Expressions of strength 

e IRAT/ TRAT 6—Dilution and concentration 

e IRAT/ TRAT 7—Infusions and IV admixtures 

e IRAT/ TRAT 8—Chemical equivalency 


This assessment sequence matched online modules 2 through 9. 


Score data from lecture exams, homework modules, and lab RAT for all students was 
collected and compiled for statistical analysis. Correlations and tests for significance 
among score categories were determined using Excel™ (Microsoft Corporation) 
software, in conjunction with XLSTAT™ (Addinsoft Inc.) and StatPlus™ (AnalystSoft 
Inc.) software add-ons. A threshold of a = .05 was selected for tests of significance. 


FINDINGS 


Score correlations between IRAT scores and lecture course assessment scores, and 
between TRAT scores and lecture course assessment scores are presented in Table 1. 
Correlations between homework module performances and IRAT or TRAT scores are 
presented in Tables 2 and 3. To summarize and to contrast the interventions, average 
IRAT and average TRAT scores, respectively, are associated with Final Exam scores in 
Figures | and 2. 
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Spearman’s correlation coefficients were used due to detection of non-parametric 
distributions throughout data comparisons. Individual readiness assurance tests 
exhibited low-to-moderate (.2 <r < .6) correlations with applicable lecture assessment 
scores (Table 1), all in statistically significant fashion. In contrast, team readiness 
assurance tests exhibited predominantly weak (-.2 < r < .3) correlations with lecture 
assessment scores. Statistical significance was observed in the correlations between 
TRAT 1 (metric system and conversion of units) scores and both Exam 1 (p < .05) and 
Final Exam scores (p < .005), and in the correlation between TRAT 5 (expressions of 
strength) scores and Final Exam scores (p < .05). 


Table 1 
Correlations of IRAT and TRAT results with lecture assessment scores 
Exam | Exam 2 Exam 3 Chem. Equiv. Final Exam 
(68+22.6)# (66.1+22.0)# (57.1428.8)# (62.6429.0)# (79.4414. 9)# 

Trat 1 Correlation 54 Al 
p-value <.0001* .001* 

Trat2 Correlation 45 42 
p-value .00* .00* 

Trat 3 Correlation 32 31 
p-value .008* .O1* 

Trat4 Correlation 40 39 
p-value .001* .001* 

Trat5 Correlation oe 50 
p-value <.0001* <.0001* 

Trat6 Correlation Al 36 
p-value .001* .003* 

Irat 7 Correlation 28 29 
p-value .02* .02* 

Trat 8 Correlation 40 AT 
p-value .001* <.0001* 

Trat 1 Correlation 25 Al 
p-value .04* .001* 

Trat 2 Correlation 23 9 
p-value .06 12 

Trat 3 Correlation -.02 .02 
p-value 90 85 

Trat 4 Correlation 15 13 
p-value 23 29 

Trat 5 Correlation Sal 28 
p-value 39 .02* 

Trat 6 Correlation 23 22 
p-value .06 .07 

Trat 7 Correlation .03 .03 
p-value 76 -76 

Trat 8 Correlation .02 .00 
p-value 90 99 

* Correlation is significant (p < 0.05) *Mean + Standard Deviation 


Individual readiness assurance tests corresponded to practice module exercises 
addressing identical topics. Low to moderate correlations were observed between IRAT 
and practice module first attempt scores (Table 2). Statistical significance was observed 
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in the correlations pertaining to the metric system (p < .05), dosage calculations (p < 
.001), reducing and enlarging formulas (p < .05), expressions of strength (p < .05) and 
IV infusion and admixtures (p < .005). Very low-to-low correlations were observed 
between TRAT scores and module first attempt scores for all lecture topics. No 
correlations were statistically significant. 


Table 2 
Correlations of IRAT and TRAT results with practice module first scores 


Mod 2 Mod 3 Mod4 Mod5 Mod6 Mod 7 Mod 8 Mod9 
(Metric (Dose (Red. & (Spec. (Express. (Dil. & (IV (Chem. 
System) _ Calcs.) Enl.) Grav.) Strength) Conc.) | Admix) Equiv.) 


Irat 1 Correlation 29 


p-value .02* 
Trat2 Correlation 49 
p-value <.0001* 
Irat 3 Correlation 25 
p-value .05* 
Irat4 Correlation 18 
p-value 16 
Trat5 Correlation 25 
p-value .04* 
Irat6 Correlation 16 
p-value .20 
Irat7 Correlation 39 
p-value .001* 
Trat 8 Correlation -.06 
p-value 62 
Trat Correlation 7 
1 p-value 16 
Tra Correlation 23 
2 p-value 61 
Tra Correlation -.21 
3 p-value 10 
Tra Correlation -.08 
4 p-value 54 
Trat Correlation -.17 
5 p-value 18 
Tra Correlation -.05 
6 p-value sh. 
Trat Correlation .02 
7 p-value 85 
Tra Correlation -.01 
8 p-value 93 


* Correlation is significant (p <0.05) 


With one exception, IRAT scores exhibited very low-to-low correlations with module 
high scores. A statistically significant correlation was observed between data sets 
pertaining to the topic of chemical equivalency (p < .05; Table 3). Similarly, very low- 
to-low correlations were observed between TRAT scores and module high scores. The 
correlation between scores pertaining to the topic of dilution and concentration was 
statistically significant (p < .05). 
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Table 3 
Correlations of IRAT and TRAT results with practice module high scores 
Mod 2 Mod 3 Mod Mod5 Mod 6 Mod 7 Mod 8 Mod 9 
(Metric (Dose 4 (Spec. (Express. (Dil. & (IV (Chem. 
System) Calcs.) (Red. Grav.) Strength) Conc.) | Admix) Equiv.) 
& 
En.) 
Trat 1 Correlation  .21 
p-value .09 
Irat2 — Correlation .06 
p-value 64 
Trat 3 Correlation -.01 
p-value 7 
Irat4 —- Correlation .08 
p-value 53 
Trat 5 Correlation .07 
p-value 61 
Irat6 — Correlation .06 
p-value 63 
Irat7 — Correlation 23 
p-value 07 
Irat8 — Correlation 28 
p-value .02* 
Trat1 Correlation .10 
p-value 44 
Trat2 Correlation .03 
p-value 84 
Trat3 Correlation -.03 
p-value 84 
Trat4 Correlation 18 
p-value 15 
Trat5 Correlation 19 
p-value 12 
Trat6 Correlation 32 
p-value .O1* 
Trat 7 Correlation -.09 
p-value 36 
Trat 8 Correlation 19 
p-value 13 


* Correlation is significant (p <0.05) 


Figures | and 2 are scatterplot displays of the trends by which mean IRAT and TRAT 
scores are associated with comprehensive final exam performances. The plots indicated 
stronger association between IRAT performance and final exam performance when 
compared to the association between TRAT performance and final exam performance. 
The contrast in scatterplot shape was representative of the poorer correlation seen 
among TRAT and assessment scores in general. 
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Figure 1 
Scatterplot of students’ final exam scores vs. Mean IRAT scores 
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Figure 2 
Scatterplot of students’ final exam scores vs. Mean TRAT scores 
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DISCUSSION 


Although moderate in magnitude, the correlation effect sizes and statistical significance 
between IRAT scores and lecture assessment scores are distinctive. The results are 
intuitive, given that RAT and lecture assessments were similarly structured. Both sets of 
assessments were proctored and timed. In addition, the rigor of questions presented 
during the assurance tests was similar to lecture assessments. In essence, IRAT exercises 
served as a prelude to subsequent assessments in the lecture course. The relationship 
between TRAT scores and lecture assessments dramatically differ from aforementioned 
IRAT correlations, due to substantial improvements in score performances that are 
characteristic of TRAT treatments. The lower correlations associated with TRAT scores 
are perhaps indicative of team performances masking individuals’ understanding of 
topics at the time of the team encounter. Students’ IRAT scores were consistently poorer 
than TRAT scores, alluding to the beneficial nature of team collaboration to TRAT 
performance. 


Individual readiness assurance test scores better correlate to lecture assessment and first 
attempt online module scores than TRAT scores correlate to these performances. The 
highest correlations are observed between IRAT scores and lecture exams, especially 
those pertaining to the topics of expressions of strength, metric system and unit 
conversions, and dose calculations. The results infer that IRAT participation may benefit 
students’ short-term preparation for major assessments. Furthermore, IRAT scores better 
correlate with lecture assessment scores than with practice module first attempt scores. 
Although modules were made available to students at approximately the same times that 
RAT assessments were encountered, students were free to commence modules any time 
prior to the end of the semester. Some students began working on the modules on the 
date of release or (shortly thereafter), while other students opted to wait several weeks 
before starting exercises. In addition, IRAT exercises were attempted under the duress 
of controlled conditions. This contrasted with untimed, un-proctored module conditions, 
which might account for the difference seen in IRAT correlation results. 


Neither assurance test distinguishes itself from the other when correlated with module 
high scores. The lone instances of low-to-moderate correlation of assurance test scores 
with module high scores pertain to the topics of chemical equivalency and intravenous 
admixtures. Students have historically experienced difficulties with these topics, which 
are introduced in the latter stages of the course. Students have also historically exhibited 
substantial variation in performance on all lecture assessments aside from the 
comprehensive final exam. Decreased mean scores and increased standard deviations 
were observed in Exam 3 and the chemical equivalency quiz compared to other major 
assessments, indicative of the difficulties students experience with these topics. Perhaps 
these isolated instances of increased RAT correlation with module high score can be 
attributed to the challenge of these concepts. 


Statistical significance associated with moderate correlations amongst comparisons is 
intriguing. Statistical significance is limited in discerning the extent to which data 
differences are meaningful (Harlow et al., 1997). Given that the sample size is not 
especially large, the low p-values are a plausible function of effect sizes. Nonetheless, 
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the p-values provide a measure of certainty regarding the nature of assessment 
correlations. Interpretations of r-correlations differ among disciplines, particularly with 
respect meaningful outcomes. Akoglu (2018) noted that moderate r-correlations in one 
discipline are considered more impactful in other disciplines in some instances. In 
contrast to a discipline such as psychology, of which there exists a century’s worth of 
correlation reporting with well-established contexts, correlation data in pharmacy 
calculations is not as extensive. Educational studies pertaining to student achievement 
have earned a reputation for yielding small effect sizes, including correlation 
measurements (Coe, 2002). It is possible that the correlations observed meaningfully 
exemplify this trend. 


Applications of the RAT have been sparingly reported for pharmacy calculations. 
Mastropietro and colleagues (2019) reported significant improvements in calculations 
students’ TRAT scores in comparison to their IRAT scores. Wara and colleagues (2019) 
also reported use of RAT scores pertaining to pharmacy calculations, but assessment of 
quantitative ability was not the focus of the activity. However, RAT correlation data has 
been reported in medical and other pharmacy disciplines. For instance, significant 
correlations (Collins et al., 2019) and low-to-moderate correlations have been noted 
between IRAT scores and exam outcomes in medical students (Carrasco et al., 2019). 
Positive correlations have also been reported between RAT scores and learning 
outcomes in pharmacy students (Malhotra, 2019; Persky & Pollack, 2011). At minimum, 
RAT interventions might serve as harbingers of performance outcomes on subsequent 
major assessments. The noticeable contrast in correlation graph shapes between Figures 
1 and 2 suggests IRAT exposure to predictive of final exam performance, at least to an 
extent. In addition, improved performance on the comprehensive final examination in 
comparison to prior assessments implies RAT to be valuable to students’ preparation. 
The IRAT in particular appears to have been a useful way for students to practice under 
exam conditions, which might have assisted them in the long run. 


Limitations 


There were limitations to the study. The assessments of 69 students were analyzed in the 
investigation, so statistical power might have impacted tests of significance. An ideal 
experimental design would feature a power analysis prior to the commencement of data 
collection. This is difficult in an enrollment-dependent setting. Although students’ 
(verbal) feedback about their RAT experiences was mostly positive, no formal survey 
data was collected from students to gauge their opinions of the intervention. 
Furthermore, the number of calculations problems encountered during each RAT 
session (8) was a fraction of the number of problems encountered on lecture assessments 
covering identical topics. Future investigations pertaining to RAT in pharmacy 
calculations would benefit from formal collection of student feedback, and from 
standardization of assessment conditions and characteristics. 


CONCLUSION 


Although TBL and RAT interventions have been used liberally in health education, 
there have been relatively few reports of its use in teaching the subject of pharmacy 
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calculations. This study yielded low to moderate positive correlations between pharmacy 
calculations students’ IRAT scores and their conventional assessment scores. Students’ 
IRAT performances were also encouragingly predictive of their comprehensive final 
exam performances. In contrast, TRAT correlations were predominately lower in 
magnitude. Readiness assurance test results, particularly IRAT scores, may be 
associated with students’ performances on subsequent assessments in pharmacy 
calculations. The usefulness of RAT intervention within pharmacy calculation warrants 
further inquiry. 
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